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J\BSTRACT 



This final report describes activities and accomplishment of 
a 3-year federally funded project at Saint Joseph's College (Indiana) to 
better integrate undergraduate students' general education courses and 
coursework with their majors. Specifically, the college worked with all 16 
departments and 50 faculty (out of 54) to set up frequent, deliberate, and 
explicit points of collaboration between the core curriculum and students' 
majors, beginning by clarifying the structure, learning outcomes, and 
assessment plan of the majors. Then, each major's contribution to and use of 
materials from the core curriculum were planned and implemented. The study 
impacted all the college's students via the core curriculum and about 88 
percent of them in terms of their major. All the departments have improved 
structure, planned outcomes, and assessment strategies; the core curriculum 
is now perceived as the work of the whole faculty; and there is an improved 
institutional ethos of collaboration. Ongoing yearly evaluation with the 
College Student Experiences Questionnaire is expected to indicate continued 
improved student outcomes. Individual sections of the report include a 
project overview and describe the project's purpose, background and origin, 
organization, and evaluation/results. Appended materials include: a report to 
the college community; data on the project's effectiveness; and the project 
evaluation report. (DB) 
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A. Project Overview 

This three-year project was based on the assumption that the separate reform movements 
in general education and the major were misguided. Since general education and the 
major are the two principal components of every student’s one and only undergraduate 
experience, it would seem to be obvious that planned or deliberate coordination of these 
two programs would result in greater student growth and development over the semesters 
of undergraduate work. 

Saint Joseph’s College was in an exceptionally favorable position to test this assumption. 
Its general education Core — a “true core” in Sandy Astin’s terminology, where all 
students take the same 45 credit hours of courses in each of the normal (yes!) eight 
semesters — had been in operation since 1969 and had come through several serious phases 
of assessment with extremely positive ratings; and its faculty had just recently been 
involved in doing systematic reformulations of major programs, under the inspiration of 
recent publications from AAC&U. The FIPSE project, then, involved putting the finishing 
touches to the revision of majors and planning the specific interactions of Core and majors 
over the eight semesters. 

By the end of the project fifty members of the faculty (out of fifty-four full-time) and 
sixteen departments (all!) had been involved. The four departments from the first year of 
the project and the five departments from the second year all went through a two-year 
schedule of project activities. The seven departments in the third-year group did their 
work in an accelerated one-year fashion. All of that translates into having an impact on all 
of the students at the College via the Core Curriculum, and about 88% of them in terms of 
their major. The largest segments of the student body “missed” by the project are those 
undecided on a major -and the nursing students who spend half of their program off- 
campus. 



B. Purpose 

The overall goal of generating collaboration between Core and the majors was specified 
into five objectives that guided the work of each department in the project: 

1) work out an eight-semester plan for the major, distinguishing courses 
into introductory, intermediate, and advanced; 
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Project Summary 



Since the two principal components of every student’s undergraduate experience are 
general education and the major, the recent separate reform movements in general 
education (1981) and in the major (1990) should really have been one conjoined 
movement. Saint Joseph’s College worked with sixteen departments (fifty faculty) to set 
up frequent, deliberate, and explicit points of collaboration between its Core Curriculum 
and these majors. Since the Core had been in effective operation since 1969, the project 
began by clarifying the structure, the learning outcomes, and the assessment plan of the 
majors. Then each major’s contribution to and use of materials from the Core were 
planned and implemented. 



“Collaboration between General Education & the Major” 

Dr. John Nichols, Project Director 
Box 895 

Saint Joseph’s College 
Rensselaer, IN 47978 
(219) 866-6133 
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2) determine learning outcomes on each of those levels, for critical courses, 
or even for each course; 

3) set up an assessment plan that will measure those outcomes; 

4) agree on what this department’s contribution TO Core will be; 

5) decide what items FROM Core are important to this major and need to 
be highlighted and reinforced in the major courses. 

The faculty participants found that these objectives served very well to organize and focus 
their activities. The only significant problem encountered over all three years of the 
project was that of faculty turnover. Some of these departments consisted of just two or 
three people; in such a situation one faculty departure set all the work almost back to 
square one. 



C. Background & Origin 

Although many other colleges and universities have been working at revising and 
restructuring the undergraduate major, the existence and development of its Core 
Curriculum since 1969 certainly put Saint Joseph’s College in a significantly different 
situation compared to most other institutions. While working together on the AAC&U 
book Strong Foundations. Jerry Gaff made the comment to John Nichols that he was very 
disappointed that none of the institutions working on the reformulation of the major 
project had made the connection with general education. That was the precise moment at 
which this project was bom. Because of its Core program (and its recent willingness to 
restmcture majors), this was exactly what Saint Joseph’s needed to do (for its own 
advancement) and could do (for purposes of national demonstration). 



D. Project Description 

Four departments were selected for the first year of the project on the basis of being 
“easy” ones to work with — readiness to do the revisions, imaginative faculty, and 
significant numbers of majors. The other motive for starting this way was to give the PD 
the best chances for learning how to mn the project successfully. It worked out well. 

The second year brought in more complicated departments, ones that ran more than one 
major or that involved more than one discipline in the same department. The third year’s 
group was the largest, but they were also departments with smaller numbers of majors — 
and therefore less expansive programs — and we were able to mn them through the whole 
process of the five objectives in an accelerated fashion (we had learned a lot about how to 
do all of this by the third year). 

Toward the end of the project, and right before our external evaluator was going to come 
on campus for the third and final time, we brought all of the faculty participants together 
for a day-long “final exam” on the FIPSE project. The PD gave a report to the faculty on 
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what had happened in all phases of the project, and the participants worked in groups to 
finalize their action plans for the 1996-97 academic year on two points: their assessment 
plan (objectives 1, 2, 3) and their department’s specific collaboration with Core 
(objectives 4, 5). 



£. Evaluation & Results 

The assessment plan involved use of the MAPS Reading Comprehension Test, items from 
the OPI and the CPI, Pace’s CSEQ, and observations of senior performance in the Core 
10 capstone. The results thus far are encouraging — the outcomes are moving in the right 
direction — but not all that exciting... yet. The main impact of this three-year project will 
not show up for another two or three years, due to the “staging” of departments through 
the project that we did and due to the simple fact that the undergraduate experience is four 
years long. To keep faith with the original impulse behind the project, we have included 
funds for continuing all of this assessment for five more years into a new grant from Lilly 
Endowment. 

The centerpiece of this continuing project evaluation is Bob Pace’s CSEQ. We are doing 
this instrument with large samples of all four classes each year. Pace is convinced that 
selective liberal arts institutions show the “best” results on his CSEQ (The 
Undergraduates. 1990). Saint Joseph’s is a general liberal arts college (less selective). 
But if the assumption behind this whole project is true, then the collaboration that we 
achieve between general education and the major ought to result in getting SLA results on 
the CSEQ from a GLA institution! That’s our “acid test” of the project. 



F. Summary & Conclusions 

All the majors that participated in this project are in far better shape than they were before. 
The work that faculty did with the first three of our five objectives clarified many things 
about their major programs, and it also gave them a great deal of professional confidence 
in what they were doing. The Core Curriculum has been renewed in the eyes of the 
faculty too; because of the multiple ways in which we tied it into the work of the majors, 
it simply cannot be seen as “foreign” to the work that any professor does in any of our 
major departments. Thus, the more or less intangible outcome of this project on this 
campus is the development of a faculty-wide ethos of collaboration. The proverbial 
“bottom line” of the project from the faculty’s point of view — from the student point of 
view it’s the CSEQ “acid test” mentioned above — may well be that the AAC&U call for a 
“revival of the responsibility of the faculty as a whole for the curriculum as a whole” 
(IntegritvJ has come true on this campus. 
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A. PROJECT OVERVIEW 



This project was conceived and implemented as an attempt to generate as much 
collaboration as possible, and for as long as possible, between general education and the 
major at Saint Joseph’s College in Indiana. Saint Joseph’s is a liberal arts institution with 
a high percentage of traditional-age students and also a high percentage of residential 
students. 95% of the students graduate in the time-frame of eight semesters. The College 
has had the same Core Curriculum — the same 45 hours of classes taken by all four-year 
students! — in operation since 1969, and the faculty had been working on “reformulation” 
of its major programs with AAC&U publications for three years prior to the start of this 
FIPSE project in the fall of 1993. 

Thus, one can see that Saint Joseph’s, was in an unusually favorable position to work at 
coordinating the two rhythms of growth and development that exist in the student’s 
undergraduate experience: general education and the major. The hypothesis that was to 
be tested in the project is that such coordination would lead to more student growth and 
development, over eight semesters, than the usual situation where general education and 
the major are left in splendid isolation from one another. Some data from Astin’s recent 
book indicated that “true cores” (where all students take the same courses) do have an 
impact, and our own evaluations of our Core program (most noticeably in FIPSE’ s 
“National Project IV” twelve years ago) came up with solid evidence of significant impact 
from the Core over four years; so we began the project with high confidence that 
whatever coordination we might achieve between these two patterns of growth and 
development would certainly show up in multiple measures of outcomes from freshman to 
senior years. 

Since the Core Curriculum was so well established and was showing up in assessment as 
doing what it was supposed to do, our project began with the majors. We began by 
completing the recent work on “reformulation” of the majors that the College had started 
in 1990 in the course of making preparations for a North Central re-accreditation visit. 



The project ultimately involved sixteen academic departments and fifty members of our 
faculty over its three years. The numerical analysis of that statement is on the next page of 
this narrative. The only segments of our student body that were “missed” by the project 
were those undecided about a major and those in a nursing program that is operated 
through a contractual agreement with St. Elizabeth’s Medical Center in Lafayette, Indiana. 
Therefore, 73% of our total student body (950 count) was affected by the project but 88% 
of our on-campus student population. 
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FIPSE Participarion (3 years) 



Team 


Dent. 


Faculty 


Sum 


Majors t%) 


Sum 


Alpha 


Comm. 


4 




4.89 




93-94 


Educ. 


4 




6.85 




4 


Hist. 


3 




3.15 






Psych. 


3 


14 


6.85 


21.74 



Beta 


English 


3 


2.39 




94-95 


Mgmt-Mrkt 


3 


14.24 




5 


Math-Physics 


6 


2.50 






Phil-Rel 


4 


0.76 






Pol Sci 


2 18 


3.48 


23.37 



Gamma 


Acct. 


3 


5.11 




95-96 


Biol. 


3 


10.33 




7 


Chem. 


3 


0.98 






Comp Sci 


3 


2.28 






Music 


3 


1.52 






PE 


1 


4.67 






Soc-Hum Serv 


2 18 


3.05 


27.94 



Totals 


50 




73.05 


3 years 
16 


out of 54 FT 


RN 


16.74 






Undecided 


8.48 






“Covered” 


98.27'! 



9/6/96 
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Assessment of the impact of the project was carried out during each of the three years. 
The obvious complicating factor is that the full impact of what we did will not show up 
until two or three years after the end of the three years of the project. Therefore, the most 
important assessment work is yet to come, so we have incorporated a continuation of all 
of the assessment work in our FIPSE plan into a Lilly grant that will run from 1997 until 
2001. We will be able to do all the assessments we did from 1994 through 1996 with 
FIPSE funding again from 1997 through 2001 with Lilly funding. Within the next five 
years, then, we should have thorough documentation on what has been achieved and, 
more importantly, how to make the impact more effective. 



B. PURPOSE 



The overall defining goal for this project was to achieve collaboration, throughout all eight 
semesters of the normal undergraduate experience, between our general education Core 
and the majors that students select. For years we had seen the relationship between Core 
and the majors as complementary — but not yet collaborative. However, since the students 
were all the same and most of the faculty were the same, such collaboration obviously did 
occur. The most accurate statement of our goal for the project, then, would be that we 
proposed to switch from occasional, individual, and serendipitous collaboration to 
continuous, collegial, and structured collaboration. The potential had been there since 
1969, the way this Core Curriculum is set up; all we had to do was make it happen. 



The single goal was specified into five objectives that guided the work of each 
department that participated in the project: 

• 1) work out an eight-semester plan for the major, distinguishing courses 
into introductory, intermediate, and advanced; 

2) determine learning outcomes on each of those levels, for critical courses, 
or even for each course; 

3) set up an assessment plan that will measure those outcomes; 

4) agree on what this department’s contribution TO Core will be; 

5) decide what items FROM Core are important to this major and need to 
be highlighted and reinforced in the major courses. 
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The faculty found that this set of objectives worked very well to focus and organize their 
participation in the project. The five objectives also made it very easy to gather the 
participating departments together in plenary sessions to share insights and ideas about 
every one of the five objectives. These plenary sessions turned out to be one of the most 
productive facets of the project — after departments had done enough work on their own 
to give them something to show off to others — and they had the effect of bringing 
“lagging” groups up to the level of the best performers. 

We also learned, by starting the project with four “easy” departments in 1993-94, how to 
work with the five objectives most productively. Each objective was translated into a 
slightly different but more psychologically productive question: 

1,2,3) Picture your majors as they go across the stage at Commencement. 
What do you want to see in them? What would make you proud? How 
do you get them to that point? And how will you know it? 

4) What should every graduate from this College know about your 
discipline? (No one said “no” to this; thus, all had to contribute TO 
Core.) 

5) What help can student growth and development in Core furnish to 
student growth and development in your major? 

The single biggest problem encountered in this project was beyond all doubt — personnel 
changes. When you are working with two or three person departments, one departure 
can throw two years of work back to square one. Unfortunately, that happened with four 
of the sixteen departments in our project. This turnover wasn’t fatal to the project, but it 
certainly complicated the work. 



C. BACKGROUND & ORIGINS 



Saint Joseph’s College is located in rural northwest Indiana. It has an enrollment just shy 
of 1,000 students with most of them traditional age and 75% of them in residence on 
campus. Non-traditional students are in a nursing program that is half off-campus and in 
the teacher training program. Admission to the College is only mildly selective, but top 
scholars are recruited so the student body runs the whole range of SAT and high school 
grades. The curriculum is structured such that a Core segment is in every one of the eight 
semesters, which means that students may start their major in the first semester of the 
freshman year. Significant majors (in terms of size) at the College are biology, biology- 
chemistry, psychology, political science, management, marketing, education, and 
communications. 
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The Core Curriculum has been in operation since the fall of 1969. It consists of 45 credit 
hours, six each semester with a three-hour seminar in the second semester of the senior 
year. The deliberate message that lies in this structure is that general education is every bit 
as important as the major and is not something to get out of the way. In fact, the College 
holds that there is as much a cumulative aspect to learning in general education as there is 
in the major. General education is also a “study in depth.” Every one of the Core 
segments is interdisciplinary and team-taught, with the result that faculty at the College are 
habitually interacting with colleagues from other departments on a day-by-day basis. As 
the single most visible academic operation on campus, the Core program has attracted a 
lot (more than $4M since 1976) of grant funding to the College — and a lot of faculty 
development is required to run this program — and it has likewise attracted a lot of 
evaluation too — Core tends to be praised or blamed for most things that happen on 
campus! The diagram on the next page illustrates the structure and the content of the 
Core Curriculum, a structure that has been almost totally constant since 1969 and a 
content that varies considerably from year to year (this may be a good hint on how to get 
stability in “gened”). 

The proximate source for the idea behind this project was the work that was done, with 
inspiration from AAC&U publications, on “reformulating” the undergraduate major. We 
began this work in about 1990, as we did our self-study for a North Central visit. While 
working on the Strong Foundations book, Jerry Gaff happened to comment to the Project 
Director (John Nichols) that one of his disappointments with the AAC&U project on the 
major was that no institution seemed to make any connections with general education in 
doing all of this restructuring. Click! That was exactly what Saint Joseph’s was on the 
verge of doing and needed to do (for its own advancement) and could do (for national 
demonstration). Thus, the idea for the project came into focus: connect the rhythm of 
growth and development in Core with that in each major, coordinate cumulative learning 
in both endeavors, and end up with greater student growth and development at the end of 
four years. 

One of the neat things about this project, from the faculty’s point of view, was that they 
got special stipends for doing work on their majors. For years we had been running 
faculty and curriculum projects connected with the Core, but this one started with getting 
their major in shape — a welcome change! It also made use of all the previous projects we 
had run for the Core program, so there was a great deal of “already accumulated capital” 
that could be invested in this project. The outside consultants brought in for each major, 
as it did its restructuring work, provided an input of fresh ideas to people who had been 
hearing the same ideas from the same colleagues for years. The single most important 
external support for the faculty came from these disciplinary consultants. The Project 
Director got a lot of help from the PD meetings, but the faculty got the most help from 
these consultants. 
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D. PROJECT DESCRIPTION 



We worked with four departments in the first year of the project: Communications, 

Education, History, and Psychology. These four were selected because the PD judged 
that they would be easy to run through the process (the five objectives), and he could 
thereby learn how to work the process with other departments. That judgment turned out 
to be correct — on both counts. 

In 1994-95 a more complex set of departments (with more than one major in some of 
them) was taken into the project; English, Management & Marketing, Math-Physics, 
Philosophy-Religion, and Political Science. Both the “alpha” and this “beta” groups had 
two years of activity in the project, but the final “gamma” group only had one academic 
year’s worth of project-funded activities, so their process was accelerated in comparison 
with the first two groups: Accounting, Biology, Chemistry, Computer Science, Physical 
Education, Sociology-Human Services, and Music. 

The work of the project for each department consisted of doing what was required to 
respond to the five objectives discussed in section “B” above, and in the order specified by 
the numbering. The first two groups of departments had two years of project activities in 
which to accomplish this: 

August — an orientation workshop; organize work on five objectives 
First academic year — bring in a consultant to help with structure of major 
May-June — work out full responses to five objectives; 
three departments did a total restructuring; 
three departments did substantial restructuring; 
the other ten did what might be styled “fine-tuning.” 

Second academic year — implement the new structure, outcomes, and 
assessment 

May — revise the whole plan for better implementation the next year 

In addition to the clarity and focus that the five objectives gave to each department’s 
work, the faculty participants found that the type and scheduling of activities suited the 
work they had to do -very well. All of the participants found the project very well 
organized and conducted. We had no problems with these aspects whatsoever; to repeat, 
out biggest problem was in faculty turnover. The faculty was ready to do this work; they 
had years for Core development work behind them. They were overjoyed to be able to 
get to work on their majors (for a change). And the two dimensions fit together admirably 
well over the course of the three years of the project. 

On August 19, 1996 we held a final plenary session for all the faculty from all the 
departments that had participated in the project. The first hour of the morning session 
featured a report to the whole group (which constituted 95% of the College’s faculty) 
from the PD on what had been accomplished overall during the three years of the project. 
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Then the groups worked on putting the final touches on objectives four and five: how, in 
concrete doable terms, was their department this year going to contribute TO the Core 
Curriculum and make use of materials FROM Core in its major? In the afternoon, 
another three hours were spent in putting final touches to each department’s assessment 
plan (who? what? when? how?), with departments working in groups that seemed to have 
similar approaches to ways of doing assessment (e.g., all those using portfolios worked 
together to polish up that process). We called this day-long event our collegial “final 
exam” for the FIPSE project. The eight-page packet of materials from this day is 
included in this report as Appendix A. 



E. EVALUATION — PROJECT RESULTS 



The plan that was followed for project assessment was identical to that submitted to Dora 
Marcus right after the fall, 1993 PD meeting, with one exception. We did four CAAP 
tests in the first year of the grant but did not repeat them in the next two years. One 
reason was the cost of these instruments (looking ahead to a necessary continuation of this 
evaluation after the end of the project period), and another reason was that we could not 
keep copies of the instruments in order to see exactly what students were not getting right 
in terms of the outcomes that we were interested in measuring. Therefore, we decided 
that, although CAAP tests gave us comparisons with national data, those tests were of no 
use to us for the primary purpose of assessment, namely to find out exactly what 
improvements we needed to make. 



The assessment plan, for the rest of it, provided and will continue to provide us with 
plenty of data to make good judgments about the impact of our project. Here are the 
measures that were used in all three years of the project and which will be continued for 
five more years: 

MAPS Reading Comprehension 

Given to all freshmen at Orientation and repeated for a 33% sample 
of sophomores at the end of their fourth semester. 

Selection of OPI and CPI questions 

Following Chickering’s lead, we identified 73 items from these tests 
that deal with intellectual and attitudinal traits that are shared 
outcomes for the Core and the majors (e.g., theoretical orientation, 
complexity, tolerance, altruism). This survey is given to 50% 
samples at the end of the freshman and the senior years. 
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Pace’s CSEQ 

We have data from this instrument going all the way back to 
FIPSE’s “National Project IV,” making both institutional and 
national comparisons possible. Students also find that it is a good 
exercise for them to take stock for themselves of what they are 
gaining from their undergraduate experience. We give the CSEQ to 
50% samples of all four classes at the end of each academic year. 

Core 10 Observations 

Three retired faculty members, all very familiar with the Core 
program, observe the senior capstone seminar presentations of 40% 
of the students and rate them on research skills, oral skills, ethical 
and interdisciplinary acumen. 



The main problem with our assessments, as previously mentioned, is that the main results 
of our project will show up in the years after the FIPSE funding is over. Therefore, we 
have secured funds from Lilly Endowment, as part of another project, to continue all the 
components listed above until the spring of 2001. That will give us wonderful longitudinal 
data on the impact of our project, for the freshmen that we had at the beginning (fall, 
1993) will be graduating in the spring of 1997. The years after that will be even more 
interesting, as more and more departments started collaborating with the Core in the 
second and third years of our project. We will be able to produce each year from 1997 
through 2001 a packet of assessment data similar to the 34-page packet from spring of 
1996 (and the same sort of packet was submitted to FIPSE at the end of each of the first 
two years of the grant) that is included as Appendix B in this report. 



Bob Pace’s 1990 book on the CSEQ, The Undergraduates, gave us the summary outcome 
for this project. In that book he demonstrates how selective liberal arts institutions show 
the “best” results on his CSEQ. Saint Joseph’s College is a general liberal arts institution 
(less selective), but the collaboration between general education and the majors that we 
accomplish here should promote unusual student growth and development over eight 
semesters, so that this GLA school ought to show SLA results on the CSEQ! That is the 
“acid test” that we have adopted for evaluation of the impact of this project. We’ll keep 
you informed. 

The results that we have thus far can be summarized in the following fashion: 

MAPS — Sophomores seem (third year compared to first) to be making 
more progress than they did previously. As we get more 
departments (this year and hereafter) reinforcing the skills 
development from Core in the introductory courses in their 
disciplines, even more progress should show up. 
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OPI-CPI — There is much more desired change on both the intellectual and 
the attitudinal measures than there was back in National Project IV 
and at the beginning of this project. 

CSEQ — The results are mixed, with some great changes and some 

“mysteries.” With the Lilly funding we can enlarge our samples, 
thus getting better longitudinal data to make this most critical 
analysis. 

Core 10 - In 1996, the data varied quite a bit according to the professor 
involved. Now that the faculty has been alerted to this assessment, 
future observations ought to provide more accurate outcome data 
on what students can do in this capstone experience. 

The external evaluation of the project was done at the end of each of the three years by 
the same person. Dr. Janice Green, so that we could have an external longitudinal 
judgment of the ebb and flow of the project. Dr. Green was Vice President at Bradford 
College for Arthur Levine and now, in retirement, is a consultant for the New England 
Research Center on Higher Education in Boston (U. Mass.). Her nine-page final 
evaluation is attached as Appendix C. 

Project dissemination has thus far taken the form of contributions to annual conferences of 
professional organizations (AGLS and AAC&U), and a new mini-workshop on 
collaboration between general education and the major run by the PD John Nichols was 
added to the workshops on general education that Jerry Gaff runs for AAC&U (1995, 
1996, 1997). In addition, Nichols included presentations on this project while doing 
consulting work at four institutions the past three years. 



F. SUMMARY & CONCLUSIONS 



There are four claims that are justified by our experiences throughout this project and 
supported by the assessment data that have thus far been generated: 

1) All of the majors that participated in this project are in far better shape than 
they were prior to the project. They have made great progress in clarity of 
structure, in deliberateness of planned outcomes, and in doable assessment 
strategies. The pressure that will be needed to keep all of this continuously 
implemented will come from the North Central Association, because when they 
return in 2001 they will expect to find years of “used” assessment data. 
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2) The Core Curriculum has benefited by being perceived to a degree not 
previously attained as the work of the whole faculty — because it is now closely 
tied to the growth and development in the student’s major. Core carmot be seen 
by any department as in any way “foreign” to the work that it does with its majors. 

3) What has come out of the project for the College as a whole is an institutional 
ethos of collaboration. Though more intangible than the other claims, this one is 
stronger; in fact it’s the most emphatic outcome of the project. In its strongest 
form, we would claim that Saint Joseph’s College is one institution where the 
AAC&U call for a “revival of the responsibility of the faculty as a whole for the 
curriculum as a whole” (Integrity) has been realized. 

4) There are two additional insights gained during the project that may be true of 
the national scene in higher education: 

a) The more attention given to assessment, the more required courses 
seem to be structured into a major program; 

b) The more collaboration between general education (especially in a 
“core”) and the major is stressed, the more iimovative and outcomes- 
oriented does teaching in the major become. 




H 
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Information for FIPSE 



The most important (and most obvious) assistance from the Fund is of course the 
recognition that a locally developed idea has enough merit to be funded for 
implementation. The facts that FEPSE grants are highly competitive and are awarded only 
after a strenuous process of peer review serve to reinforce that impression. Therefore, not 
only money but a certain amount of “clout” comes to an institution along with a FEPSE 
grant. 

I myself, as PD, did not have to use that “clout,” because I had been VPAA here for 8 
years and had also conducted many faculty and curriculum development grants connected 
with our Core Curriculum. But it was very interesting to see how colleagues used the 
FEPSE sponsorship when they took their plans for restructuring their majors before our 
Academic Cabinet. “But this is what we worked out in the FEPSE project!” This “clout” 
can be used to promote both good and bad ideas, but it is a fact that it exists. 

The most assistance that we got for our project came from Dora Marcus, Joan Krejci, and 
Frank Frankfort. Joan and Dora wouldn’t let me stop short on our assessment plan, so we 
ended up with a real good one. Joan made me explain all the local jargon and elliptical 
expressions, so that what we were doing became clearer not only to her but to ourselves. 
Frank is an old fiiend (I’m old; he’s not), and he helped me out of a real bind at the end 
when my assessment data and my final external evaluation weren’t getting done before the 
official end of the project (he processed an extension). 

But the PD meetings were also an important factor in getting our project on track and 
moving to a successful (we all think— here!) completion. In the first year, I worked and 
worked on the assessment plan (no carousing that year in D.C. — besides, the zoo???). 
The second year, Joan put me on an assessment panel and Rusty Garth recruited me for a 
workshop on leadership. Both of those events forced me to really get my head together 
on those two topics, and it did me a lot of good as far as keeping on top of our own 
project. I missed the final year’s meeting because of a conflict with a Board of Trustees 
meeting here, but it gave the College the opportunity to send another person to D.C. to 
see what’s happening in higher education. That turned out to be very fortunate for SJC, 
because that person has just become our new VPAA. 

These are the thoughts that I have right now. I hope they are helpful. I intend to keep 
sending assessment materials to Dora, so if I think of anything else I’ll pass that along too. 



THANK YOU! 
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APPENDICES 



The following packets of information are included with this report and 
are integral elements of this final report. 



Appendix A: Report to SJC Community 

(Agenda for “final exam” day, 8/19/96) 



Appendix B: Data fi*om 4/96 FIPSE assessments 



Appendix C: Final external evaluator’s report 

Dr. Janice Green 
(rec’d 10/15/96) 
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Report to S.TC Community on FIPSE Project 

8/93 to 5^96 

John Nichols, Project Director 



INTRODUCTION: Objectives for the Project 



(1) An 8'semester plan for the major 

All participating departments clarified the structure of their major(s) at least to the point of 
distinguisUng introductory from intennediate from advanced courses. Some changed 
their programs to various degrees (“Catalog changes”), while others clarified structure 
without modifying it. 

Qarification 
Educ 
Hist 

&igl 

Math 



Acct 

Biol 

Chem 

Soc 

CS 



(2) Specification of student outcomes 

All participating departments specified learning outcomes for students at least in terms of 
the 3 levels.(above), some by critical courses, and some for each required course. 



Some Change 
C&TA 



Mgmt 

Mrkt 

Phil-Rel 



Lot of Change 
Psych 



Pol Sci 



Hum Serv 



(3) An assessment plan for #2 

All departments developed something for this third objective, but there is a general need 
for more specificity and concrete planning. (And that’s why we’re here on 8/19!) There 
are good elements in every department’s plan, so we can improve all the plans by sharing 
ideas (this afternoon). 
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(4) The major’s contribution TO the Core program 

What follows is my summary of the key things judged by each department as its 
contribution to our collegial general education program. As a summmy (and by me), 
each entry loses a lot of its richness. Emendations are welcome... 



Educ reading skills 

C&TA speaking skills (2 handbooks); theatre 

Hist content in 1-4, 7-8; critical thinking; use of evidence 

Psych listening skills; complexity of human behavior; respect for 

individual differences 

Engl writing skills; analysis & appreciation of literature 

Mg^t & Mrkt knowledge of world of business 

Math critical thinking (quantitative); math proflciency 

Phil-Rel critical thinking (deductive); ultimate concerns; synthesis 

Pol Sci political theory; comparative government; policy analysis & critique 

Acct economic (tax) policy 

Biol & Chem scientific method; environmental & ecological values 

S oc social construction of reality; raising of consciousness beyond the 

individual to the group 

Hum Serv commitment (“realistic”) to human dignity 

C S Netscape; spreadsheets; Info Age 



(5) What the major uses FROM Core 

Same comments, as in #4, about the following sununaries... “Skills” is omitted, because 
it rather obviously applies to everyone. 



Educ content of courses are correlated with Core for all 8 semesters 

C&T A ♦♦ Core prerequisites for courses in major (skills, content) 

Hist pedagogy; complementary content (art, literature, philosophy, reli gion) 

Psych parallel growth (Perry) between Core and major 

Engl complementary content (history, art, literature in other cultures) 

h^mt & Mrkt ' breadth; knowledge of the world 
Math history & science for the capstone course 

Phil-Rel complementary content (NW religions, art, literature, history) 

Pol Sci complementary content 

Acct breadth, knowledge of the world 

Biol & Chem complementary content & methods of thought 

Soc complementary content 

Horn Serv complementaiy content (grounding of values) 

C S breadth; knowledge of the world 
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AGENDA (morning. 8/19/96^ 



Collaboration between the Majors and Core 



(1) Objective 

Evidently, since we all have the same students and the Core faculty is about 75% the same 
as the faculty for the majors, a lot of connections will occur “naturally.” What the FBPSE 
project is all about, however, is turning individual, random, serendipitous, and occasional 
events of collaboration into widespread, purposeful, planned, and even institutionalized 
collaboration. Just to start the discussion, let’s simply consider the possibilities as far as 
skills development (thinking, speaking-listening, writing-reading), the content of courses, 
and the complementarity of the disciplines are concerned. 



(2) Some examples 

It would certainly seem possible for us to reinforce the skills development pursued in 
Cores 1 and 2 by means of the assignments given in the freshman courses in the majors. 



[** Syllabus materials] 

Core asks that students adopt the format for their papers from their m aj or ; so 
there is one practical matter of collaboration. One following page has an 
outline of the many steps involved in a “research paper,” and if we cover all 
of these things (incrementally!) both in Core and the majors seniors should 
have the process well under control. 

On the intermediate levels (SO, JU) we have opportunities to make use of content from 
Cores 3-4 (the classics in art, philosophy, and literature; the Scriptures) and the ways of 
looking at the world from other disciplines (the “other” cultures; natural science; plus the 
disciplines involved in Cores 3-4). The majors run these Cores, so the contribution of the 
majors TO Core is (almost) well under control; but departments can do a world of good 
for the College and for their students by showing interest in these complementary subject 
matters and intentionalities. “You may be going to be an accountant (programmer, 
doctor, etc.), but this is important stuff for you to know. You’re expected to know it to 
graduate from SJC, and that’s precisely your differential advantage in the job market!” 



Core 1 (961 . 20 pages) 
memoir (narrative)’"’" 
summary 
position paper’"’" 
compare/contrast 



Core 2 (952; 25-30 pages) 
two research papers 
summary 
position paper 
annotated bibliography 



The evaluation of Core 10 presentations last year looked at seniors’ research and 
presentation skills, their ability to do an interdisciplinary project, and their abiity to handle 
the ethical dimensions of their topic. The same things are applicable to capstone 
experiences in the major. AAC&U sets knowledge of the history and ethics of a discipline 
as the assessment criterion for seniors. If we all value collaboration, seniors ought to be 
expected to use material from other disciplines to clarify and to reinforce their research in 
a major (and some of our majors, such as business, are going more and more into cross- 
functional analysis). And capstone courses already use research and oral presentation 
skills as primary criteria for grading. 



(3) Our task 

What, very specifically, will be the collaboration between our department’s major and the 
Core Curriculum in 1996-97? 



Attachment 



Departmental Action Plan 



FIPSE project "final" 
8-19-%, A.M. 



Collaboration: Core < — > Maior 
Contributions TO Core Use FROM Core 



SKILLS: 



CONTENT: 



METHODS: 



Dept. 

Chair 



THE WRESEARCH” PAPER (CorftB 1-10) 



Purpose; type of paper (see taxonomy) 
audience 

scope, medium, formality 

sxibject > topic > thesis] 

brainstorming 

looping 

grid 

pentad 

controlling purpose POCUS** 



sources; general knowledge 

first-hand experience 
exploratory search in Lib. 
annotated bib . 
evaluate sources: la, 2a... 
working bib.** 



strategy; outline** 

assemble/select sources 
REFOCUS 
argument 

coherence: order/logic of devel. 
spatial 
chronological 
specificity 
alternation 
climax 
feuailiarity 

notes 
final bib. 



CO 

z 

o 

HH 

CO 

CO 

u 

CO 



CO 

UJ 

oc 



CJ 

UJ 



CO 

CJ 

z 

HH 

Q 

< 

UJ 

CC 



UJ 

cc 

o 

CJ 



Draft; quotations 

document at ion 






Product ; rewrite: the whole-organization 

paragraphs — unity, topic 
sentences & words— effective 
edit: spelling, punctuation, mechanics 



’' Instruction ” 
clear guidelines 



Library 

Reference books 
On-line catalog 
Per. Indexes 
Gov . doc . 



notes 

summarize 

paraphrase 

quote 

plagiarism 



format 



**Grade 



{**Feedback} 



J 



AGENDA (afternoon. 8/19/96> 
Assessment 



(1) North Central Association on assessment 

Higher education in general and our accrediting association in particular have become very 
sophisticated and more and more demanding about assessment over the last ten years. 
When NCA returns here in 2000-01, the visiting team will expect to find us with years of 
assessment data — documented! — as well as evidence of how we have been using these 
data. To clarify... 

WHY do assessment? To improve teaching and learning. 

WHAT is assessed? Student academic achievement (“outcomes”— #2 objective) 



At the March, 1996 NCA meeting I was given a copy (and “found” 5 more to bring back 
to campus) of a new booklet on assessment. It is a summary of their expectations of us, 
of every accredited institution in the region. What sorts of things will constitute 
“acceptable” measures of student academic achievement at NCA? 



Direct Measures 
pre- and post-tests** 
capstones 
standardized tests 
licensure exams 
local tests 

essays (blind scored 
by “others”) 
senior projects 
(int. & ext. juries) 
exhibitions, performances 
(video’s, audio’s) 
internship evaluations 



Indirect Measures 
surveys 
exit interviews 
studies of grads 
retention 
time to degree 
SAT, ACT 
graduation rates 
job placement 



Non-Measures 
curr. review 
NCA,NLN... 
faculty: 

publication 
recognition 
study abroad 
enroll, trends 
diversity 
grades, GPA 



(2) Core Curriculum 

The assessment plan for Core is attached on a following page. 



(3) Neat ideas from your own FIFSE work 
External assessment 

Acct. — CPA exam (others have NTE, MCAT, LSAT, some GRE’s...) 
Hist — survey of grads who teach history 




2 ? 



Clarity 

Chem — diagram of the structure of the major 
Engl “ excellent 6-point description of the graduate 
common 4-year text (and test) for all majors 
Psych — 4 X 4 matrix: key courses, key outcomes 

Specificity within a plan 

Hum Sery - Perry schema for student growth (and instructors) 
Educ — 8-semester, i.e. longitudinal, assessment plan 
Mgmt “ use of the capstones 
Mrkt “ pre- and post-test 

Math “ check-sheets on outcomes from indiyidual courses 

Biol - specially designed assignments that “recap” most of a course 



(4) Our task 

a) Get specific (like we tell students on essay exams)! 

Releyant data »> lots of data. 

What? (deriyes from Why?) - data releyant to outcomes 
When? — all courses? key courses? which? 

Who? - whole department 
How? “ details, details... 

Where? — keeping of records 

b) How will the department use the assessment data to improye student 
academic achieyement? 

c) Focus on 1996-97; you’ll reyise the plan for the following years. 



Attachment 



Departmental Assessment Plan 



FIPSE project "final" 

8-19-%, P.M. 



Assessment of Student Outcomes 
Internal; What? Which Outcome? Who? How? 
FR 



SO 



m 



SE 



Extemafc 



How we will USE tiiese data^ 



Where? 



Dept. 

Chair 



S' 



Core 



FR 1 



2 



SO 3 



4 



JU 5,7 
6,8 



SE 9 



ASSESSMENT of CORE 



Orientation 
MAPS (aU) 



OPI-CPI (75) 

Cognitive 

Attitudinal 



CSEQ (60) 
Activity scales 



MAPS (75) 



CSEQ (50) 
Rating of SJC 



CSEQ (40) 
Gains 

OPI-CPI (60) 



10 



CSEQ (40) Observation 
Seminar pres. 

- interdiscpl 

- ethics 

- research 

- oral skills 
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September 6, 1996 

To: Academic Cabinet, Core Directors, and selected IPPC members 

From: John Nichols 

Re: Data from 4/96 FIPSE assessments 



There are four groups of data in this packet: 

a. Bob Pace’s College Student Experiences Questionnaire 

Assesses how students are making use of the College, 
and with what results. Covers all 4 years. Need a copy? 

b. MAPS (don’t ask!) Reading Comprehension Test 

Progress from FR Orientation to end of SO year. 

c. Selected (43) questions from OPI and CPI 

Deal with cognitive and affective traits allegedly 
connnected with Core. 

d. Observation of Core 10 seminar presentations 

Our capstone course: ergo, “what have we wrought?” 

INDEX of CONTENTS 

a. CSEQ -- overall institutional effectiveness 
General 

demographics on the 4/96 data 1 

student academic engagement 2 

student ratings of the College environment 3 

Activity Scales (what students spend time on...) 

4/96 SJC data: FR — > SE 4 

SJC data compared to national data, FR --> SE 5 

SJC data compared to national data, by school type 6 
graphical presentation 7 

Estimates of Gains (what they judge they’ve gained) 







SJC 4/96 data: FR — > SE 


8 






highest gains and lowest gains 


9-10 






SJC data compared to national FR-->SE data 


11 






three years of SJC data 


12 






SJC data compared to national data, by school type 


13 






graphical presentations 


14-15 


b. 


MAPS — 


reading progress from FR — >S0 


16-19 


c. 


OPI -CPI 


items -- FR to SE changes 


20-31 


d. 


Core 10 


“capstone” observations 


32-34 



With a little digging, there is a wealth of information contained in this 
packet — some to cheer about; some to help us do better. 



O 

ERIC 



jT 



iL 
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CSEQ data from April, 1996 



Demographics of the Sample 



Item 


Total 


FR 


SO 


JU 


SE 


Number of students 


204 


49 


49 


55 


51 


Age: =< 22 


185 


42 


45 


54 


44 


23-27 


12 


5 


1 


0 


6 


>= 28 


7 


2 • 


3 


1 


1 


Male 


116 


29 


30 


22 


35 


Female 


88 


20 


19 


33 


16 


Transfer 


27 


3 


5 


8 


11 


Commuter 


23 


8 


7 


5 


3 


Part-time 


2 


0 


1 


1 


0 


Minority 


24 


8 


6 


3 


7 



The Instrument 

The College Student Experiences Questionnaire was “invented” by Robert Pace at UCLA’s Center for the 
Study of Evaluation. It has recently moved, after Bob’s retirement, to Indiana University. (And one of 
our much earlier grads, Nick Vesper, handles the data processing for the CSEQ now.) The norms for the 
CSEQ have emerged out of its use with over 3Q,000 students, and Bob Pace has published an excellent 
analysis of what can be learned from it (The Undergraduates. 1990, UCLA). 

Pace’s guiding philosophy is the radical (??) idea that what students accomplish in the college years 
depends on what they actually do at an institution then! In more sophisticated jargon, it’s a “time on 
task” or a “quality of effort” approach. The main part of the CSEQ asks students to indicate what use they 
have made of institutional resources during the academic year they’re just finishing. There are 14 of these 
“Activity Scales” dealing with everything from Library to Courses to Dorms to Clubs to Athletics. Each 
of the scales describes ten ways of students interacting with that part of the college — and the 
sophistication of such interaction increases from #1 to #10. Interaction with Faculty, for example, ranges 
from “talking with a prof after class” to “published an article with a prof” Students indicate whether they 
have done a particular item “never, occasionally, often, or very often.” 

Aside from the holistic view that this gives of how students are interacting with the college, the national 
norms for the CSEQ enable us to make comparisons with both “general” (GLA) and “selective” (SLA) 
liberal arts institutions. The pages that follow provide you with just the “tip” of the CSEQ data “iceberg” 
that we have available. I have computer print-outs and 3.5” diskettes with our data from ‘94, ‘95, and 
‘96. Anyone who wants to look into any of these data any further is welcome to borrow them. Please 
continue reading... 
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CSEQ data from April, 1996 



STUDENT ACADEMIC “ENGAGEMENT” 



FR SO JU SE Total 

Hours per week on schoolwork 



50 or more 


2.0% 


2.0 


7.3 


5.9 


4.4 


about 40 


22.4 


16.3 


27.3 


11.8 


19.6 


about 30 


42.9 


42.9 


27.3 


33.3 


36.3 


about 20 


26.5 


20.4 


32.7 


33.3 


28.4 


less than 20 


6.1 


18.4 


5.5 


15.7 


11.3 



Readine — textbooks 



>20 


28.6 


22.9 


28.3 


5.9 


21.4 


10-20 


38.8 


20.8 


45.3 


29.4 


33.8 


5-10 


14.3 


25.0 


22.6 


47.1 


27.4 


<5 


14.3 


22.9 


3.8 


15.7 


13.9 


none 


4.1 


8.3 


- 


2.0 


3.5 



Reading — non-assigned 



>20 


- 


2.0 


- 


2.0 


1.0 


10-20 


6.4 


12.2 


5.6 


3.9 


7.0 


5-10 


12.8 


12.2 


7.4 


13.7 


11.4 


<5 


42.6 


40.8 


57.4 


49.0 


47.8 


none 


38.3 


32.7 


29.6 


31.4 


32.8 



Writing — essay exams 



>20 


18.4 


2.1 


16.7 


16.0 


13.4 


10-20 


38.8 


29.2 


31.5 


32.0 


32.8 


5-10 


34.7 


35.4 


31.5 


32.0 


33.3 


<5 


6.1 


31.3 


20.4 


16.0 


18.4 


none 


2.0 


2.1 


- 


4.0 


2.0 



Writing — papers 



>20 


18.8 


16.3 


16.7 


15.7 


16.8 


10-20 


29.2 


36.7 


40.7 


37.3 


36.1 


5-10 


37.5 


28.6 


37.0 


27.5 


32.7 


<5 


10.4 


14.3 


5.6 


17.6 


11.9 


none 


4.2 


4.1 


- 


2.0 


2.5 
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CSEQ data from April, 1996 



STUDENTS RATE THEIR COLLEGE 



There are four tables in the CSEQ results that ask students to rate their institutions. Three of them are 
directly from sets of questions in the instrument: rate the college on a 7-point scale in 8 different respects; 
indicate how much you like college; and tell whether you’d go to the same college if you were to start all 
over again. The fourth table is called a “satisfaction index” (1 = low; 8 = high), a score that is calculated 
by the computer in terms of the responses to the other questions. Here are the SJC results. 





FR 


SO 


JTJ 


SE 


SJC 












Rate SJC on 7-pt. scale (1 = low) 
(#’s = average ratings) 


academic 


5.6 


5.5 


5.5 


5.5 


5.5 


aesthetic 


4.9 


4.7 


4.6 


4.7 


4.7 


critical 


5.2 


4.9 


5.2 


5.2 


5.1 


vocational 


5.0 


4.6 


4.6 


4.5 


4.7 


pers. relevance 


5.3 


5.2 


4.9 


5.3 


5.2 


rel. w. students 


5.3 


5.5 


5.9 


5.7 


5.6 


rel. w. faculty ** 


5.9 


5.5 


5.7 


6.0 


5.8 SE>SO 


rel. w. admin. 


4.8 


4.6 


4.7 


4.4 


4.6 


Do you like college? 


no 


4.1% 


8.2 


1.9 


5.9 


4.9 


neutral 


16.3 


12.2 


20.4 


9.8 


14.8 


yes 


53.1 


38.8 


42.6 


37.3 


42.9 


enthusiastic! 


26.5 


40.8 


35.2 


47.1 


37.4 


Would you go to SJC again? 
deHnitely no 


14.6% 


14.3 


3.7 


17.6 


12.4 


probably no 


14:6. 


16.3 


24.1 


9.8 


16.3 


probably yes 


43.8 


44.9 


40.7 


27.5 


39.1 


deHnitely yes 


27.1 


24.5 


31.5 


45.1 


32.2 



Satisfaction index (#’s = # of students) 
Low 1 


_ 




_ 


_ 


0 




2 


1 


4 


1 


3 


9 




3 


5 


1 


- 


2 


8 




4 


5 


4 


4 


5 


18 




5 


7 


6 


16 


5 


34 




6 


14 


14 


12 


9 


49 




7 


8 


11 


8 


8 


35 


High 


8 


9 


9 


13 


19 


50 




34 
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CSEQ data from April, 1996 



14 Activity Scales 






Scale 


FR 


SO 


JU 


SE 


SJC 


Significant fa). .05 


Library Experiences 


20.1 


19.6 


21.1 


22.9 


20.9 


SE > FR,SO 


Exper. with Faculty 


22.5 


21.1 


23.0 


24.3 


22.8 


SE> SO 


Course Learning 


26.2 


24.5 


27.7 


27.6 


26.6 


JU,SE > SO 


Arts 


18.3 


18.2 


20.7 


19.2 


19.1 


JU>FR,SO 


Student Union 


24.1 


21.8 


24.5 


23.5 


23.5 




Athl/Rec Facililties 


26.2 


24.9 


24.6 


24.3 


25.0 




Clubs & Organiz. 


19.0 


20.0 


23.3 


21.3 


21.0 


JU>FR 


Exper. in Writing 


25.9 


22.8 


24.9 


25.7 


24.8 


FR,SE > SO 


Personal Exper. 


20.7 


22.1 


23.8 


22.6 


22.3 


JU>FR 


Stud. Acquaintances 


26.1 


25.4 


29.3 


28.8 


27.5 


JU,SE > FR,SO 


Science & Technology 


17.7 


18.3 


20.3 


18.6 


18.8 




Dorms 


25.0 


25.6 


30.1 


26.3 


26.8 


JU > FR,SO,SE 


Topics of Conversation 


21.2 


20.5 


23.4 


24.0 


22.3 


JU,SE > FR,SO 


Info in Conversations 


13.7 


13.0 


14.7 


15.8 


14.3 


. JU,SE>SO; SE>FR 



What do these numbers mean? 

Most of the Activity Scales in the CSEQ contain descriptions of ten different ways in which a student can 
interact with the institution in that particular area. On each of the ten items the student may respond with 
“never, occasionally, often, or very often.” Scoring is done by assigning point values from 1 to 4 to each 
answer to each item — with “never” getting a 1 and “very often” getting a 4. Therefore, scores on each 
Activity Scale may theoretically range from 10 (all “nevers”) to 40 (all “very oftens”). 

Since items 7-10 on each scale get fairly sophisticated, scores from the 30,000+ students who have taken 
the CSEQ tend to average out in the low 20’s range. There are a couple of scales that get lower than that 
(e.g.. Science) and a couple that get higher (e.g.. Course Learning). In a sense, therefore, you may look at 
the numbers as GPA’s multiplied by ten, if you keep in mind that C and C+ are very good scores on this 
particular instrument. (The numbers actually are averages from total points on a particular scale divided 
by the total number of students who took the CSEQ. So, it is a score, and not a percentage.) 

What gets interesting in Bob Pace’s book is when he compares average scores on these scales among 
different types of institutions: research universities (RU), doctoral universities (DU), comprehensive 
colleges and universities (CCU), general liberal arts colleges (GLA), and selective liberal arts colleges 
(SLA). The table on the next page is the first bit of evidence for his claim that “it is the selective liberal 
arts colleges that have the best results” (Pace, 1990, p. 61). More evidence is to come. 
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CSEQ data from April, 1996 

The last section in the College Student Experiences Questionnaire asks students to estimate how much 
they have gained on 23 different items. Possible answers are on a 4-point scale: very little (1), some (2), 
quite a bit (3), and very much (4). The data below take the responses and calculate a “GPA” for each 
estimate of gains. 

An entry that ranks higher than 3.00 means that students estimated that they gained “quite a bit” or “very 
much” on the aspect of growth and development in that item. 



Question 


FR 


SO 


JU 


SE 


“Sienificant” ((a). .051 


Vocational 


2.59 


2.41 


2.66 


2.67 




Specialization 


2.65 


2.59 


3.00 


2.88 


JU > FR,SO 


Broad gen. ed. 


2.59 


2.58 


3.00 


3.16 


JU,SE > FR,SO 


Career 


2.81 


2.77 


3.08 


2.84 




Arts 


1.86 


2.10 


2.19 


1.94 




Literature 


2.18 


2.18 


2.40 


2.18 




Writing 


2.78 


2.55 


2.87 


2.90 


JU,SE > SO 


Computers 


2.84 


2.88 


2.90 


3.12 




Other philosophies 


2.55 


2.64 


3.13 


3.27 


JU,SE > FR,SO 


Own values 


2.71 


2.59 


3.00 


3.25 


SE>FR,SO; JU>SC 


Self-understanding 


3.00 


2.84 


3.17 


3.20 


SE>SO 


Underst’d others 


2.96 


2.86 


3.19 


3.18 




Function on team 


2.78 


3.06 


3.15 


3.12 


JU>FR 


Health habits 


2.39 


2.52 


2.45 


2.44 




Underst’d science 


2.00 


2.15 


2.45 


2.45 


JU,SE > FR 


Underst’d technol. 


1.94 


2.02 


2.34 


2.18 




Conseq. sci. & tech. 


2.06 


2.27 


2.55 


2.63 


JU,SE > FR 


Analytic thinking 


2.67 


2.61 


2.91 


3.06 


SE > FR,SO 


Quant, thinking 


2.31 


2.54 


2.57 


2.60 




Put ideas together 


2.61 


2.86 


3il 


3.22 


SE>FR,SO; JU>FR 


Learn on own 


2.77 


2.96 


3.19 


3.22 


JU,SE > FR 


Import of history 


2.65 


2.61 


2.83 


2.75 




Knowledge of world 


1.96 


2.21 


3.25 


3.24 


JU,SE>FR,SO 


# of 3. 00+ 


1 


1 


11 


11 




# of “firsts” 


0 


1 


10 


13 




# < 2.00 


3 


0 


0 


1 





J. Nichols, 7/96 
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CSEQ data from April, 1996 



‘Highest” gains estimated (>= 3.00) 


3.27 


Other philosophies SE 


3.25 


Own values SE 


3.25 


ICnowledge of world JU 


3.24 


Knowledge of world SE 


3.22 


Learn on own SE 


3.22 


Put ideas together SE 


3.20 


Self-understanding SE 


3.19 


Learn on own JU 


3.19 


Understanding others JU 


3.18 


Understanding others SE 


3.17 


Self-understanding JU 


3.16 


Broad general education SE 


3.15 


Function on team JU 


3.15 


Put ideas together JU 


3.13 


Other philosophies JU 


3.12 


Computers SE 


3.12 


Function on team SE 


3.08 


Career JU 


3.06 


Analytic thinking SE 


3.06 


Function on team SO 


3.00 


Broad general education JU 


3.00 


Specialization JU 


3.00 


Own values JU 


3.00 


Self-understanding FR 



Tally: FR 1 

SO 1 

JU 11 
SE 11 



J. Nichols, 7/96 
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CSEQ data from April, 1996 



“Lowest” gains estimated (<2.00 or low 2’s) 

1.86 Arts FR 

1.94 Understanding technology FR 
1.94 Arts SE 

1.96 Knowledge of world FR 

2.00 Understanding science FR 

2.02 Understanding technology SO 

2.06 Consequences of science & technology FR 

2.10 Arts SO 

2.15 Understanding science SO 
2.18 Literature FR, SO, SE 
2.18 Understanding technology SE 



Tally: FR 6 

SO 4 
JU 0 
SE 3 



J. Nichols, 7/96 
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CSEQ data from April, 



1996 

STUDENT ESTIMATE OF GAINS 



The last section in the CSEQ asks students to estimate how they progressed in 23 different respects up to 
the present time in their college careers. Pace examines what percentages of students estimate that they 
have made “substantial gains’" in some respect, and he defines that as the sum of the “very much” and the 
“quite a bit” responses, leaving “very little” and “some” out of the calculation. What follows are SJC 
students’ estimates of what they have gained out of college, as students have responded to the CSEQ in 
the last three years (FIPSE project). The numbers are substantial gains, namely the sum of two 
percentages, as defined above. 





4/94 


X 


4/95 


X 


4/96 


X 


Intellectual: analytic 


68% 




71.9 




67.7 




quantitative 


44 




49.2 




47.7 




synthetic 


68 




72.5 




70.8 




inquiry 


76 


64 


81.6 


68.8 


75.6 


65.5 



Science: 


science 


31 




35.1 


39.3 




technology 


23 




30.8 


30.7 




consequences 


32 


29 


35.1 33.7 


41.3 37.1 



GenEd: 


breadth 


64 




69.2 


67.7 




arts 


16 




24.8 


26.7 




literature 


37 




36.2 


35.6 




writing 


72 




73.0 


64.3 




philosophies 


66 


51 


69.2 


54.5 66.5 52.2 



Personal/social: values 


72 




69.2 




67.3 




self-underst, 


74 




78.4 




76.3 




underst others 


79 




74.0 




75.8 




team 


70 




71.9 




70.1 




health 


44 


68 


48.7 


68.4 


50.0 


67.9 



Vocational: 


job 


54 




58.4 


56.5 




specializ. 


64 




70.2 


65.8 




career 


76 


65 


75.1 


67.9 71.5 64.6 



“new”!! 


computers 


64.9 


69.0 




history 


61.1 


58.4 




kn. of world 


52.5 


59.5 55.3 60.9 
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Students Reporting Substantial Progress 
Toward Types of Goals/Outcomes 



Percent 

80-H 



70- 



60- 



50- 



Personal/. 

Social 



Intellectual 

Skills 



GLA 


67 


SIA 

ecu 


64 


DU 


61 


RU 


60 



SLA 


70 


GIA 


67 


DU 


RU 


64 



General 

Education, 




40- 


RU 43 


Science 


Goa 


Is 




RU 


36 






SLA 


34 


30- 




ecu 


30 






DU 


28 



Vocational 

Goals 



GLA 27 



GLA 


66 


DU 


63 


ecu 


58 


RU 


54 


SLA 


52 



Difference between highest and lowest type 
7 23 9 



14 
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MAPS Reading Comprehension 

College Board [top score = 45] 






Purpose : 

This test is administered each year at Freshman Orientation by the Counseling Services Office to all 
new freshmen. As part of our FIPSE project, we arranged to repeat the test with sophomores at 
the end of Core 4. Although we did it in each April of the FIPSE project, we were only able to 
match scores between entering freshmen and fourth-semester students in the last two years. 



Hypothesis : 

IF we can get collaboration between freshman courses in the majors and the first Core segments, 
THEN students ought to show more positive gains on this test than before the FIPSE project. The 
precise collaboration would lie in giving the same writing assignments in Cores 1-2 and in the 
introductory courses in the majors: summaries, two-reason arguments, and short critiques. (Save 
the “term papers” for later!) These assignments call into play the same skills as tested by the 
MAPS: analyze a text, distinguish main points from supporting evidence, and so forth. 



Application : We gave it to 

4/94 entered 8/92 

4/95 entered 8/93 

4/96 entered 8/94 



Presentation: 



Core 4 sophomores in each year of our project: 

unaffected by FIPSE collaborations 

one year of collaboration (37 matched scores) 

two years of collaboration (46 matched scores) 



1) Scores from the 3 testings, with Means and (^uartiles. 

2) Change in scores of individual students between Orientation and end of Core 4. 

3) Changes occurring at different levels of scores. 



Interpretation : “It almost worked right!” 

The second-year group (8/93—4/95) gained almost a full quartile between its two testings, much 
more than the previous year’s students had gained. The tWrd-year group (8/94—4/96), however, 
showed much less positive difference in the gains that students made. Obviously, instead of... 

year 1 < year 3 « year 2 (how the data turned out) 

we should have had... year 1 < year 2 < year 3 (in the best of all possible worlds). 
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17 



Part of the reason why this occurred may emerge from the way the data are presented in display 
#2. Although most students were motivated to do well in the repeat of the MAPS--so they could 
see how much they had improved over four semesters of college !--fully twelve of the 46 students 
with matched scores in 4/96 got a lower score in 4/96 than they did back at Freshman Orientation 
in 8/94. The suspicion is that something went wrong with the administration of the test to 
probably one of the four Core 4 discussion groups in April of 1996. The 17 or 18 students in one 
discussion group would be a large enough group to skew the data. 



Despite the doubts about the 4/96 data (if it’s not taking the test seriously that’s involved)-or the 
real frustration and depression (if that’s the case!)— something very significant did show up in 
the data. Clearly (first two years) and fairly clearly (third year), some good things happen in the 
freshman year, because transfer students have lower scores in Core 4 than those who started out as 
freshmen at the College (Core 1). 



Conclusion : 

We need more years of data (derived from controlled administration of the MAPS). The test takes 
only 20 minutes, is easy to score and to process, and students c^ be motivated to do their best on 
it, both at Orientation and later as sophomores. We need data at least up to April, 20(X). 



More departments will be involved in our Core-major collaborations in 1996-97; in fact, we should 
cover virtually all freshman courses next year. So the original hypothesis can be subjected to 
ongoing test. With more extensive collaboration on basic skills development among all the 
professors teaching freshman Core and introductory courses in the majors, students ought to show 
much bigger gains on the MAPS Reading Comprehension Test than they did in any of the 
preceding years. This testing will also serve as part of our assessment program for North Central, 
so the data will continue to be gathered. 
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MAPS results from initial three years of the FIPSE project 

Apr 96 





Aug 92 


Apr 94 


Aug 93 


Apr 95 


Aug 94 


45 


0 


0 


0 


1 


1 


44 


4 


0 


2 


2 


6 


43 


3 


3 


5 


4 


7 


42 


4 


3 


9 


^ 8 


8 


41 


9 


1 


10 


4 


12 


40 


18 


*5 


15 


3 


11 


39 


15 


6 


^ 15 


X 4 


20 


38 


*24 


3 


17 


4 


11 


37 


12 


6 


19 


5 


25 


36 


17 


X 1 


19 


o 4 


X 19 


35 


18 


4 


X 23 


0 


13 


34 


X21 


o 8 


12 


1 


10 


33 


16 


2 


15 


1 


13 


32 


18 


0 


10 


0 


12 


31 


14 


2 


11 


1 


O 6 


30 


15 


1 


° 8 


0 


® 10 


29 


O 10 


1 


9 


2 


10 


28 


14 


0 


9 


1 


7 


27 


10 


1 


6 


0 


6 


26 


11 


0 


1 


0 


5 


25 


9 


0 


5 


0 


3 


24 


6 


2 


3 


1 


3 


23 


4 


0 


3 


0 


3 


22 


5 


0 


4 


0 


2 


21 


2 


1 


1 


0 


0 


20 


2 


0 


2 


0 


1 


19 


3 


1 


1 


0 


0 


18 


2 


0 


1 


0 


0 


17 


0 


0 


- 0 


0 


0 


16 


0 


0 


0 


0 


0 



8/92 


8/93 


8/94 


N = 287 FR 


N = 229 FR 


N = 


X = 34 
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OPI - CPI questions 

1976-80 and 1996 data 



Instruments : Omnibus Personality Inventory, California Personality Inventory 



History of use 

Both of these instruments were used in their entirety gn campus during RPSE’s “National 
Project IV: Examining the Varieties of Liberal Education” (1979-1981), and 43 true-false questions 
out of the hundreds in the two instruments seemed well crafted to gather data about some of the 
important objectives of the Core Curriculum. Since Arthur Chickering had made so much use of 
the OPI in his Education and Identity . there was a significant precedent for using these items from 
these tests in doing assessment of what happens to students in the SJC Core Curriculum. Thus, 
we have responses from freshmen and seniors at SJC from back during NP4 and from the recent 
years of our 1993-96 FIPSE project. 



Presentations 



There are many different avenues of analysis possible with these data. The ones that were 
judged to have the most bearing on the current FIPSE project have been pursued and the results 
summarized on the following pages. Further possible steps in analysis are mentioned below. 

1) Text of the 43 questions selected from the OPI and the CPI. 

2) Scores of students (64 FR, 38 SE) who did the test in April, 1996. 

2.1) Raw scores, i.e., number of items answered “as desired” (as “keyed”). 

2.2) Scores grouped in sets; mean, median, mode, percentiles 

3) The 25 items that showed biggest change (in %) between 1996 FR and 1996 SE. 

4) Comparison of scores (% as keyed) between 1976-1980 and 19%. 

5) Freshman-to-Senior percentages of students answering as keyed, 1976-80 and 19%, 
with text of questions. 

6) Items showing most change, 1976-80 compared with 19%. 

7) Traits showing more change in 19% than back in 1976-80 and how much more: 



Thinking Introversion 


+ 13.2 percentage p>oints 


Responsibility 


+ 11 


Sense of Complexity 


+ 10.4 


Altruism 


+ 8.1 


Theoretical Orientation 


+ 3.4 
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Conclusions 



1) Students in 1996, distinct groups of freshmen and seniors, showed more “intellectual 
development” (as defined by three OPI-CPI traits) than freshmen in 1976 compared with seniors in 
1980: 

Thinking Introversion (TI): 13.2% more average gain 

Sense of Complexity (Co): 10.4% more average gain 

Theoretical Orientation (TO): 3.4% more average gain 



2) Students in 19% showed more progress in development of “attitudes” which the 
College views as positively connected with its mission than did students in the 1976-80 samples: 

Responsibility (Re): 1 1.0% more average gain 

Altruism (Al): 8.1% more average gain 



Continuation of Assessment 



Now that all the “bugs” have been worked out of the computer processing of this 
assessment technique, these OPI-CPI items can be repeated for 25-33% samples of freslmen and 
seniors with little expenditure of either student or faculty time/effort. Four different levels of 
analysis can be applied very easily — and with useful results: 

1) Continue FR--SE comparisons as regards current students on campus. 

2) Compare FR— SE differences from one year with previous years. 

3) Determine which items show most differences from year to year. 

4) Analyze data from the point of view of the personality traits included. 



Although we haven’t yet pushed the analysis to discussion of results on individual items, it 
is this level of analysis that should prove to be most interesting as we continue to study the 
collaboration of particular disciplines with the general education Core. At the very least, the data 
on some of the items ought to provoke good discussions among groups of faculty, as well as with 
focus groups of students. 



Another possible future assessment study that might be fruitful would be to compare the 
freshman OPI-CPI responses with the CIRP data that we have on entering freshman classes. 
(How much do they change in their first year at the College?) One could also compare seniors in 
19% with 1980 and freshen in 19% with 1976 and continue those comparisons year by year. 
(Astin does this for the CIRP each year, but we could extend the study into the college years.) 



We will continue to administer this instrument and analyze the results for at least the next 
four years. The freshmen who come to Saint Joseph’s College in August, 19% are the ones who 
will benefit from the full impact of our FIPSE project, so we are most interested in seeing their 
responses to these items in April, 1997 and then again in April, 2000. (We’ll keep in touch!) 
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(J) CORE EVALUATION QUESTIONS 

Students, please help us get some indication of the impact of the Core program 
by giving us your honest true or false response (A for true and B for false on 
the Scantron form) to each of these questions. We appreciate your help. 

Dr. John Nichols 



1) I would like to leam more about 
the history of human thought. 

2) I take an active part in group or 
class discussions. 

3) It is highly unlikely that astrology 
will ever be able to explain anything. 

4) Usually I prefer known ways of doing 
things rather than tryingout new ways. 

5) I prefer having a principle or theory 
explained to me rather than attempting 
to understand it on my own. 

6) I am uninterested in discussions of 
the ideal society or Utopia. 

7) I like short, factual questions in an 
examination better than questions that 
require the organization and interpre- 
tation of a large body of material. 

8) I want to know that something will 
really work before I am willing to 
take a chance on it. 

9) I am aroused by a speaker's descrip- 
tion of unfortunate conditions in a 
locality or country. 

10) I don't blame anyone for trying to 
grab all he can get in this world. 

11) I dislike assignments requiring 
original research work. 

12) I tend to ignore the feelings of 
others when accomplishing some end 
that is very important to me. 

13) I don't like things to be uncertain 
and unpredictable. 

14) I would rather remain free from 
commitments to others than risk serious 
disappointments or failure later. 

15) I like to fool around with new ideas 
even if they turn out later to have 
been a total waste of time. 

16) I discuss the causes and possible 
solutions of social , political, 
economic or international problems. 

17) I enjoy listening to debates and 
discussions on social, economic, or 
political problems. 

18) I like assignments which require 
me to draw my own conclusions from 
some data or a body of facts. 

19) I am more interested in the application 
of principles and theories than in the 
critical consideration of them. 

20) Science has its place, but there are many 
important things that can never possibly 
be understood by the human mind. 

21) I would enjoy writing a paper explaining 
a theory and presenting the arguments 
for and against it. 

22) I am bored by discussions of what life 
will be like 100 years from now. 

23) It doesn't bother me when things are 

O uncertain and unpredictable. 

ERIC 



24) Husbands, rather than wives, should 
have the final say in family matters. 

25) I dislike test questions in which the 
information being tested is in a form 
different from that in which it was 
learned. 

26) I would enjoy studying the causes of 
ah important national or international 
even and writing a paper on them. 

27) I have difficulty imagining the 
reaction of a person of another period, 
race or country, to a given situation 
or environment. 

28) I much enjoy thinking about some 
problem which is a challenge to the 
experts* . 

29) After a lecture or a class I think 
about the ideas presented. 

30) The thinking which I do is largely 
limited to that which I must do in the 
course of my work. 

31) I expect that ultimately mathematics 
will prove more important for mankind 
than will theology. 

32) I like to look for faulty reasoning 
in an argument. 

33) Our thinking would be a lot better off 
if we would just forget about words 
like "probably," "approximately," and 
"perhaps . " 

34) For most questions there is just one 
right Answer, once a person is able to 
get all the facts. 

35) Maybe some minority groups do get rough 
treatment, but it's no business of mine. 

36) We ought to worry about our own country 
and let the rest of the world take care 
of itself. 

37} I like poetry. 

38) Every citizen should take the time to 
find out about national affairs, even 
if it means giving up some personal 
pleasures . 

39) I like to read about history. 

40) Only a fool would try to change our 
.American way of life. 

41) I like science. 

42) Society owes a lot more to the 
businessman and the manufacturer 
than it does to the artist and the 
professor. 

43) I like to read about science. 
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Items with "biggest" (>10 points) change between FR and 

(P 

Item Trait 


SE 

Change - 


< 

% by SE % by FR 


36) We ought to worry about oi r own country 


Re 


48 


82 


34 


and let the rest of the world take care 
of itself. 

26) 1 would enjoy studying the causes of 


Am 


32 


76 


44 


an important national or international 
event and writing a paper on them. 

1 7) 1 enjoy listening to debates and 


Tl, Am 


29 


82 


53 


discussions on social, econimic, or 
political problems. 

25) 1 dislike test questions in which the 


Tl 


26 


42 


16 


information being tested is in a form different 
from that in which it was learned. 

3) It is highly unlikely that astrology 


TO 


25 


50 


25 


will ever be able to explain anything. 

38) Every citizen should take the time to find 


Re 


23 


84 


61 


out about national affairs, even if it means 
giving up some personal pleasures. 

24) Husbands, rather than wives, should 


Am 


22 


89 


67 


have the final say in family matters. 
14) 1 would rather remain free from 


Am 


21 


82 


61 


commitments to others than risk serious 
disappointments or failure later. 

1 0) 1 don't blame anyone for trying to 


Am 


20 


50 


30 


grab all he can get in this world. 

6) 1 am uninterested in discussions 


Tl 


19 


74 


55 


of the ideal society or Utopia. 

21) 1 would enjoy writing a paper explaining 


TO 


19 


66 


47 


a theory and presenting the arguments for 
and against it. 

30) The thinking which 1 do is largely limited 


Tl 


19 


63 


44 


to that which 1 must do in my work. 

8) 1 want to know that something will really 


TO, Co 


18 


74 


56 


work before 1 take a chance on it. 

22) 1 am bored by discussions of what life 


TO 


17 


84 


67 


will be like 1 00 years from now. 

28) 1 much enjoy thinking about some problem 


To 


17 


84 


67 


which is a challenge to the experts. 

27) 1 have difficulty imagining the reaction 


Tl 


16 


82 


66 


of a person of another period, race or 
country, to a given situation. 

11)1 dislike assignments requiring original 


TO 


15 


76 


61 


research work. 

7) 1 like short, factual questions in a test 


Tl 


14 


47 


33 


rather than ones requiring organization 
and interpretation. 

18) 1 like assignments which require me to 


TO 


14 


87 


73 


0 ! my own conclusions from facts. 
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40) Only a fool would try to change 
American way of life. 
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43) 1 like to read about science. 
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1 2) 1 tend to ignore the feelings of others 
when accomplishing some important end. 
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84 


73 


13) 1 don't like things to be uncertain 
and unpredictable. 
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47 
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32) 1 like to look for faulty reasoning... 
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the history of human thought . 

2 _ I take an active part in group or T 
class discussions. 

^ It is highly unlikely that astrolo^ T 
will ever be able to explain anything. 

j Usually I prefer known ways of doing F 
things rather than trying out new ways . 

^I prefer having a prin. or theory F 
explained to me rather than attempting 
to understand it on my own. 
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I like short, factual questions in a F 
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. I want to know that something will F 
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locality or country. 

I don't blame anyone for trying to 
grab all he can get in this world. 

I dislike assignments requiring 
original research work. 



I tend to ignore the feelings of 
others when accomplishing some end 
that is very important to me. 

I don't like things to be uncertain 
and unpredictable. 
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, After a lecture or a class I think 
about the ideas presented. 
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The thinking which I do is largely 
limited to that which I must do in 
the course of my work. 


F 


75 


69 


44 


63 


TO 

Am 


, I expect that ultimately math F 

^ will prove more important for mankind 
than will theology. 


68 


82 


66 


68 


TO 


I like to look for faulty reasoning 
in an argument . 


T 


79 


65 


84 


76 


Ai 


jj Our thinking would be a lot better 
^ off if we didn't use "probably," 
"approximately," and "perhaps." 
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For most questions there's one 
right answer, once a person gets 
all the facts. 
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... We ought to worry about our country 
^ and let the rest care for self. 
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36) We ought to worry about our own country 
and let the rest take care of itself. 
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30) The thinking 1 do is largely limited 
to my work. 
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6) 1 am uninterested., ideal society... 
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8) 1 want to know something will work... 
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29) After a lecture 1 think about Ideas... 
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Core 10 Seminar Observations 

30 assessments, April ^96 
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Core 10 



This semester segment is the capstone course for the Core Curriculum. It is usually taken in the 
eighth semester, although some exceptions have to be made for three-year programs and for certain 
internships. Hence, it is in this seminar course that we should be able to observe “the best” 
performance of our students. 

Students are required to present their Core 10 seminar research in both written and oral form. Their 
seminar paper must involve at least two different disciplines, be thoroughly imbued with ethical 
reflection, and show senior level research skills. The key importance of this paper for the FIPSE 
project is that student performance precisely here will manifest how much connection between the 
Core and the major has occurred for the individual student! The seminar oral presentation will be 
an occasion to demonstrate oral and critical thinking skills. 



Our Process 



Three retired professors, all well acquainted with both the Core Curriculum and the major 
programs at the College, observed ten Core 10 presentations each — therefore we have 30 different 
assessments. Due to scheduling peculiarities, professors in Core 10 were unevenly covered; ten 
students for A, only two for B, and 18 for C. 

The observers filled out a sheet with their written comments on four questions that seem to 
describe the principal outcomes from both the major and the Core program at the time of the 
seniors’ departure from the College. 

1) Was the talk “interdisciplinary?” Did the student integrate evidence or principles from 
at least two disciplines into the talk? 

2) Did the talk deal explicitly with ethical issues? How well were they handled? Were 
principles , and not just opinions, involved? Was there evidence of the student’s 
knowledge of Christian Humanist values? 

3) What was the quality of the research skills exhibited? 

4) Were the oral skills of the presenter what you would expect of a senior in Core? 



Presentation 



There is just one page of data to accompany this analysis. The Project Director debriefed the three 
observers, for the purpose of getting a feel for their overall impressions and some idea of how they 
went about expressing their judgments. Then he took their thirty response sheets and put their 
written comments on each question onto a four-point scale: 1 = Poor; 2 = OK; 3 = Very Good; and 
4 = Excellent. The four-point scale made calcination of several “GPA’s” possible — overall, for 
each question, and by teacher on each question. This operation did not misconstrue what the 
observers judged, because the overall “GPA’s” matched their impressions exactly. 
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Interpretation 



Let’s be clear that there is a double interpretation at play here: the numerical rating of the observers’ 
written comments, and also generalizations from those comments with regard to deficiencies that 
show in the Core 10 presentations. Let’s also realize that the Core 10 professors may well be able 
to finesse or even to correct some of these judgments, since they have access to the students’ full 
write-up on their research topics. Nonetheless, the observers and the Project Director made these 
judgments: 

1) The interdisciplinarity of the students’ work was disappointing (2.17/4.00), although 
the expectations of one professor (C) seemed to make a big difference. 

2) The amount and the sophistication of ethical reflection exhibited varied a great deal 
according to the professor; and the 230 overall is nothing to brag about. 

3) The observers all remarked that too many secondary sources (newspapers, television, 
and anecdotes) played too heavy a role in the student presentations, even though the 
rating on research got up to 2.^. 

4) This was the most successful (2.73) of the four outcomes, but even here there was a 
lot of room for improvement — perhaps by giving students a good model to imitate in 
preparing and malang their presentations. 



Conclusions 



The Class of 1996 has only been minimally connected with our FIPSE project and therefore also 
minimally affected by it. These data can only serve as baseline data for ongoing assessments of 
this type in 1997, 19^, 1999, and 2000. Because the Core 10 seminar presentation ranks so high 
in relevance to desired outcomes — 

a capstone performance, so it assesses students right before graduation; 

as interdisciplinary, it calls on the major and Core; 

with the paper and the talk, it recapitulates all skills development — 

the repetition of this assessment will figure prominently in what we do to finally determine what 
we’ve accomplished with the FIPSE grant. It will also be an important part of our next self-study 
for North Central. 

The hypothesis that remains to be tested in upcoming years is the following: IF we have 
achieved significant collaboration between Core and the majors through our project, THEN by 
1998 the overall ratings on these four questions should climb above 3.00 and stay consistently 
there. 



QUANTITATIVE Interpretation of Written Comments from Core 1 0 Assessors 
(by John Nichols) 
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Saint Joseph's College FIPSE Project 
Final Evaluation Report 

Submitted by Janice S. Green 



INTRODUCTION 

Saint Joseph's College has coiipleted the final year of a 
three "year project funded by the Fund for the litprovement of 
Post -Secondary Education. The project was designed to enhance 
student growth and development, both intellectual and personal, 
by iitplementing planned collaboration between a mature general 
education core curriculum and sixteen disciplinary majors. It 
aimed to achieve an integrated, coherent program of studies 
that connects student learning over foiir years between the core 
and the majors. 

Participating in the project were fifty of fifty-four 
full time faculty and seventy- three per cent of the student 
body, exclusive of those in the nursing program and those as 
yet undeclared as to major field. The project was directed by 
Dr. John P. Nichols, Professor of Philosophy and until very 
recently. Core Coordinator. Dr. Nichols holds the chair of NEH 
Distinguished Professor and has served as Vice President for 
Academic Affairs at Saint Joseph's. 

This report is based upon three annual site visits, the 
most recent taking place September 26-28, 1996, discussions 
with members of each academic department, the project director, 
and the Vice President for Academic Affairs, and examination of 
materials prepared by departments and of standardized 
assessment testing results gathered and analyzed by Dr. 

Nichols . 

The first section of the report will offer the reviewer's 
findings pertinent to achievement of project goals. The second 
section will present a number of observations and 
recommendations that may be helpful to the College as it 
pursues its objectives. The reader is asked to bear in mind 
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that the faculty of seven itajors (the Gamma group) have 
participated in the project for only one year and are therefore 
not expected to have achieved results commensurate with the 
Alpha and Beta, groups, who have been involved directly since 
1993-94 and 1994-95. 

ppnGBK.q.q Aim achievements of the project 

Over the three-year period, the Saint Joseph's faculty 
has made excellent progress toward achieving the primary goals 
of the FIPSE project, while progress must be seen as somewhat 
nnpv pn among the various departments, in part due to shorter or 
longer periods of participation, there have been solid advances 
toward the goal of an integrated, sequenced, and holistic four- 
year program of studies . Dr Nichols and his colleagues are to 
be congratulated for undertaking a coirplex and demanding task 
requiring ongoing vigilance, cooperation, and creative effort. 
When mature, their program can serve as a model of educational 
coherence for institutions nationwide. 

Principal Project Areas 

T ■ Curriculum 

The departments were virtually unanimous in declaring 
that the project had stimulated a hard look at the curriculum 
of their majors. The result has been a restructuring and 
resequencing of major courses to achieve a more coitpatible and 
effective fit with the required sequence of general education 
core courses. While this effort has focused fairly heavily on 
parallel skills development and intellectual maturation, it 
involves also, although with somewhat less emphasis, attention 
to coordination of course content. For example, the content of 
several required courses in the history major relates to that 
of Core 1-4 and 7-8. Again, the junior year focus in the 
psychology major on behavioral and social structures 
purposefully relates to Core 5-6. while numerous other 
examples might be cited, it is fair to state that greater focus 
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has been placed on the concurrent development of intellectual 
skills, e.g. written and oral communication, critical thinking, 
research techniques, ways of knowing, etc. Many exairples were 
provided of a. lieightened recognition of the need to coordinate 
student assignments by type and level to ensure steady and 
predictable skill development. For instance, the student asked 
to produce a summary in Core l and again in Pol. Sci. lOl and 
in Eng. 121 will have multiple opportunities to develop this 
particular ability. Conversely, the student will not be asked 
to write a research paper until he or she has learned to do so 
in Core. Again, the Communications and Theatre Arts major now 
requires Core 1 and 2 as prerequisites for several of its basic 
courses . 

The stimulus provided by the FIPSE project to examine 
major programs has resulted as well in redesigned or updated 
curricula. For exanple, the mathematics faculty has developed 
three new courses that include humanistic and historical 
considerations and taken steps to insure that their graduates 
are skilled in the latest technologies applicable to the field. 
They are presently working to incorporate quantitative material 
into Core courses, thus assisting students to understand and 
evaluate the data with which we are daily bombarded. The 
effort of incorporation is apt to be a gradual one, as many 
faculty feel inadequate to deal with quantitative elements. 

From the perspective of Core, it was evident that the 
project has encouraged Core faculty to enrich their courses 
with related content provided by guest lecturer colleagues . 
Exanples include a lecture in Core 2 on the economics of 
slavery by an accounting professor and materials on Africa 
provided by a member of the education department. Such 
inclusions promote cross-fertilization between Core and the 
majors enhance student understanding of the connectedness 
of knowledge. 

In short, the FIPSE project has been successful in 
stimulating and motivating faculty to think about curriculum 
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with a fresh focxis and to purposefully strive for a coherent 
educational experience. 

II. Community Building 

It is generally assumed that small liberal arts colleges 
are, by virtue of size and purpose, models of academic 
community. In fact, to quote Gershwin, "It ain't necessarily 
so." Communities don't just happen; they are built and 
nurtured and, in the best of worlds, preserved. At Saint 
Joseph's, the FIPSE project has been the driving force behind a 
greatly enhanced sense of community and collegiality among 
faculty. Awareness of and satisfaction with this advance was 
expressed almost unanimously by those with whom I spoke. I 
learned of more frequent interaction between departments, 
between Core and non-Core faculty, and between individual 
faculty members . Exchange of information and ideas • is taking 
place, both formally and informally. Groups across disciplines 
are working together on curriculum and faculty development 
projects. Plans are under way for a series of informal faculty 
dinners and brown bag discussions. Most significant, there is 
a strong awareness of the need to continually seek ways to 
foster academic community. It is clear as well that the new 
^ nd very excellent Core Building, housing classrooms. Core 
faculty offices, and a faculty resource room plays a large role 
in providing the physical setting for ongoing interaction and 
communication. In working toward the project goals of cross" 
fertilization, integration, and coherence, the structures and 
habits of community are being realized and appreciated. It 
should be noted in this regard that the Interim Vice President 
for Academic Affairs (a search is under way for a permanent 
incumbent) is keenly aware of the ittportance of nurturing and 
building upon the advances achieved. 

III. Assessment 

The FIPSE project, from the initial stages of proposal - 
writing, included a strong emphasis on the formal assessment 
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and evaluation of student growth and development, learning 
outcomes, and academic programs. Each participating faculty 
team understood that engagement in the project mandated the 
development of. a departmental assessment program designed to 
measure, both quantitatively and qualitatively, the extent to 
which project goals were being met. It should be noted that, 
while outcomes assessment was not new to Saint Joseph's 
faculty, conprehensive programs of evaluation were not the 
norm. This, of course, is the situation at many institutions. 

On August 19, 1996, Dr. Nichols convened the entire group 
of project faculty for a "final exam." This was to be a 
summing-up, an opportunity for departments to present and 
discuss the fruits of their labors during the three-year period 
in terms of the following issues: an eight -semester plan for 
the major, specification of student outcomes, and an assessment 
plan. Worksheets were distributed in advance of the meeting for 
coitpletion by the faculty. 

In examining the resulting documents, I found mixed 
results, ranging from the very specific, thoughtful, and 
conprehensive to the somewhat cursory. Nevertheless, each 
department evidenced solid understanding and appreciation of 
project goals and efforts to achieve them. It was the 
assessment coitponent that appeared least well -developed 
overall. Many, if not most, departments had opted for 
portfolio evaluation - the collection of student work and 
related data to be used for ongoing analysis and guidance. In 
many departments, portfolios had already been initiated. 
However, ray discussions with faculty pointed to the need for 
enhanced expertise in the development and assessment of 
portfolio materials. The Core Coordinator and the Interim 
Academic Vice-President are fully aware of this need and will 
take steps to address it proirptly, perhaps inviting an 
assessment specialist to offer one or more workshops on canpus. 
Also underway is the appointment of an Assessment Committee 
charged to lead and coordinate the initiative. On the plus 
side, numerous departments plan to survey their graduates 
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periodically for feedback as to the value of their 
undergraduate education, their eitployment situation, advanced 
studies, etc. Indeed, such surveys have been administered by 
certain departments as a regular practice in past years. 

In addition to the assessment of majors, a program for 
evaluating student growth and development more broadly is in 
place. Data covering several years is available, providing a 
base for trend analysis, conparison with national norms, and 
identification of strengths and weaknesses. Dr. Nichols has 
been the prime mover of this effort. Standardized testing 
instruments include the College Student Experiences 
Questionnaire, MAPS Reading Conprehension Test, and the OPI-CPI 
Inventories. Faculty members have received a great deal of 
data conpiled and analyzed by Dr. Nichols. While admittedly 
the three years of the project do not permit definitive 
conclusions to be drawn, nonetheless, results in several areas 
show distinct patterns of growth and inprovement. They show as 
well areas of lesser strength. For exaitple, reading scores 
inprove substantially from the freshman to sophomore years; 
items quetying students as to values and ethical issues show 
that the College is fulfilling its Christian humanistic 
mission; on the CSEQ activity scales, the 1996 scores stand up 
well conpared to those of selective liberal arts colleges 
nationally. On the other hand, test items involving the arts, 
literature, and science and technology offer less satisfactory 
results; the reasons for this are as yet ^xnclear, but will be 
the subject of investigation. 

A final avenue for assessment is the public presentation 
required of students at the conclusion of Core 10, a seminar on 
selected ethical topics examined in the context of Christian 
humanism. In 1996, for the first time, three retired Saint 
Joseph' s faculty were invited to listen to and evaluate a 
sartple of student presentations. Students were rated on the 
basis of four issues: evidence of interdisciplinarity, evidence 
and discussion of ethical principles, quality of research 
skills, quality of oral skills. The acconpanying written 
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papers were not evaluated by the visitors. It is understood 
that this set of evaluation results can serve only as baseline 
data, since the class of 1996 was only itiiniitially involved in 
the FIPSE projject. Additionally, the assessment was 
experimental as to format and process. Nevertheless, the 
findings were less positive than would have been hoped. Given 
that Core lO is seen as a capstone experience wherein students 
can demonstrate interdisciplinary knowledge and understanding, 
ethical growth, and development of intellectual skills, it will 
be important to further explore the procedures and future 
results of this element of assessment. 

OBSERVATIONS AND SUGGESTIONS 

It is clear to the reviewer that the Saint Joseph' s 
faculty has accomplished a great deal in the past three years. 
They have worked purposefully and with good will to meet the 
goals of the FIPSE project. They recognize that full 
achievement of those goals will require ongoing commitment and 
effort. Presently, however, they express considerable anxiety 
and feelings of stress. The reasons for their discomfort may 
not be unusual in these difficult days, but they are 
nonetheless troublesome. During the past two or three years. 
Saint Joseph's has experienced an imusual rate of faculty 
attrition. This year alone a dozen new faculty arrived to take 
the place of those who resigned. The problem is a low salary 
schedule. People are forced to seek other positions for 
economic reasons. The results: a loss of programmatic 
continuity; the burden of mentoring and training new faculty to 
teach in the Core; the likelihood that new faculty will seek 
further curricular revision; and, the consequent difficulties 
in successfully pursuing project objectives. This year, for 
exanple, the psychology department (Alpha group) , which had 
produced an excellently redesigned program of studies, is 
staffed entirely by new faculty members. Those faculty who 
remain on board are increasingly burdened and see no light at 
the pnd of the tunnel . It can only be hoped that the Board of 




76 



Trustees and the senior administration can find ways to iii 5 )rove 
salaries and achieve greater faculty stability. 

It is clear to the Saint Joseph' s faculty and academic 
administration that a major effort of the next several years 
must be in the area of assessment. Far less clear at this 
point are the processes by which collected data will be 
analyzed and utilized. Such determination will be a major task 
for the new Assessment Committee and, in the light of existing 
pressures on faculty, a difficult one. In this same 
connection, it will be iit 5 )ortant, I believe, to identify the 
measures and criteria for a final outcomes assessment to 
determine whether a student has achieved, to a reasonable 
degree, the College's principal educational objectives as 
defined by the FIPSE project. Presently the student is 
required, in most majors, to take a capstone course or seminar 
in the senior year. Also required is CORE 10, mentioned above, 
and seen also as a capstone experience. The question is: how 
will each of these requirements contribute to a global 
assessment of student achievement? If they are seen as quite 
separate sources of evaluation - one of achievement in the 
major, the other of achievement through Core - it would seem 
that the holistic thirust of the FIPSE project is diminished. 
While it may be premature to raise this issue, I would urge its 
consideration as integral to the development of a conprehensive 
assessment program. 

Not surprising, but nonetheless disappointing to some, 
were the weakness of test scores in art and literature. While 
Core courses include works of literature and study of the 
visual arts in a cultural context, the English major offers 
intensive study of American and British literature, and a minor 
in studio art is required of elementary education majors, it 
appears that many students are unaffected. Given the College's 
location in a small, rural town, a stronger effort to bring the 
arts to caitpus may be at least part of the answer, e.g. 
rotating exhibits of the work of regional artists, concerts by 
visiting performers, poetry readings, etc. A donor might be 
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pursuaded to fund such a program with a small endowment . In 
addition, the faculty may consider the further inclusion of 
works of fiction in Core, not for purposes of textual analysis, 
but rather as..cultural, historical, and socio-political 
statements . 

CONCLUSION 

It has been a privilege to play a small role in the Saint 
Joseph's FIPSE project. The project is a gro\md- breaking 
initiative with iiiportant inplications for American higher 
education. A great deal has been accoitplished by a committed 
group of faculty and I am confident that their work will 
continue and thrive despite obstacles. Dr. Nichols' early 
assessment results show that progress toward stated goals has 
already been made, a fact that should hearten and encourage all 
concerned. For those of us who believe profoundly in the value 
of coherence and connectedness in undergraduate education, it 
will be iirportant to follow the Saint Joseph's project over the 
next several years and I look forward to doing so. 

I want to extend it^ thanks to Dr. John Nichols and his 
colleagues for their unfailing courtesy, cooperation, and 
hospitality during it^ cairpus visits. They have my best wishes 
for continued success in their very significant work. 
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